I 



Obion, Spivak, et al. 
703-413-3000 

4 Docket # 293352US40PCT 

Sheet 1 of 104 



FIG.1 



INPUT 
APPARATUS 



INPUT INFORMATION 
OBTAINING MEANS 



T 



101 



POINTER POSITION/ROTATION 
ANGLE CALCULATION MEANS 



104 



POINTING 
DETERMINATION 
MEANS 



X 



103 



POINTER 
GENERATION 
MEANS 



X 



105 



OBJECT 
GENERATION 
MEANS 



102 



X 



106 



DISPLAY CONTROL MEANS 



X 



1 



X 



107 



PROCESS 
CONTROL 
MEANS 



108 



STORING 
MEANS 



DISPLAY 
APPARATUS 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 2 of 104 



FIG.2 



z 
▲ 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 3 of 104 



FIG. 4 A 




FIG.4B 



Obion, Spivak, et al. 
703-413-3000 

Docket # 293352US40PCT 
Sheet 4 of 104 



FIG.5A 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 5 of 104 



FIG.6A 



301 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 6 of 104 



FIG.7 



( START ) 



DISPLAY OBJECT AND POINTER 



I 



CALCULATE POSITION, ORIENTATION, 
LENGTH ETC. OF POINTER 



I 



DISPLAY POINTER AT THE CALCULATED POSITION 
WITH THE CALCULATED ORIENTATION AND 
THE CALCULATED LENGTH 



401 



OBTAIN INPUT INFORMATION ON POINTER 



402 



403 



404 




NO 



CHANGE COLOR OF THE POINTED OBJECT AND DISPLAY IT 



406 



FIG.8A 

303a 



TP 


t 




1 


303a 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 7 of 104 



FIG.8B 



303b 




FIG.8C 



303c 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 8 of 104 



FIG.9A 




FIG.9B 




FIG.9C 



201 



301 



201 P 
303 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 9 of 104 



FIG.10A 



301 




FIG.10B 



301 




FIG.10C 



301 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 10 of 104 



FIG. 11 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 11 of 104 



FIG.13 

( START ) 



DISPLAY OBJECT AND POINTER 



401 



± 



STEP 402 ~ STEP 404 



405 



THERE IS 
OBJECT THAT IS POINTED AT 

9 

"yes 



NO 



CHANGE COLOR OF THE POINTED OBJECT AND DISPLAY IT 



IS BUTTON OF 
PEN PUSHED WHILE OBJECT IS BEING 

POINTED AT? 

YES 



407 



OBTAIN INPUT INFORMATION ON POINTER 



I 



CALCULATE POSITION, ORIENTATION, LENGTH ETC. OF 
POINTER 

CALCULATE POSITION AND ORIENTATION OF OBJECT 



I 



■DISPLAY POINTER AT THE CALCULATED POSITION WITH THE 
CALCULATED ORIENTATION AND THE CALCULATED LENGTH 
■DISPLAY OBJECT AT THE CALCULATED POSITION WITH THE 
CALCULATED ORIENTATION 



406 



408 



409 



410 



FIG.14A 



201 



201 D 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 12 of 104 



FIG.14B 




FIG.14C 




FIG.15A 



301 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 13 of 104 



FIG.15B 



301 





Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 14 of 104 



FIG.16A 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 15 of 104 

FIG.16B 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 16 of 104 



FIG.17A 

301 




FIG.17B 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 17 of 104 



FIG.18A 




FIG.18C 

201-^ r>o ^201 



Obion, Spivak, et al. 
703-413-3000 

Docket # 293352US40PCT 
Sheet 18 of 104 



FIG.18D 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 19 of 104 



FIG.19C 




FIG.19D 



201 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 20 of 104 



FIG.20 



( START ) 



DISPLAY OBJECT AND POINTER ^401 



STEP 402— STEP 404 



THERE 

IS OBJECT THAT IS POINTED 

AT? 



405 



NO 



CHANGE COLOR OF THE POINTED OBJECT AND DISPLAY IT 



406 




OPERATION FOR 
STARTING OPERATION, EDITING. AND PROCESSING 

IS PERFORMED? 

YES 



HIDE THREE-DIMENSIONAL POINTER 

I 



]-\-412 



DISPLAY PROJECTION OF OBJECT TWO-DIMENSIONALLY 
ONTO DISPLAY PLANE CLOSEST TO OPERATOR 



413 



5 



ACCEPT TWO-DIMENSIONAL GUI-LIKE 
OPERATION FOR OBJECT BY OPERATOR 



414 




NO 



415 



DISPLAY OBJECT AND POINTER 



j-^401' 



STEP 402' —STEP 404' 




CHANGE COLOR OF THE POINTED OBJECT AND DISPLAY IT| 



407 

IS BUTTON 
OF PEN PUSHED WHILE OBJECT IS BEING 

POINTED AT? 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 21 of 104 



FIG.21A 



301 




304 



FIG.21B 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 22 of 104 



FIG.22A 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 23 of 104 



FIG.23A 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 24 of 104 



FIG.24A 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 25 of 104 



FIG.24C 




304 





FIG.25A 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 26 of 104 



FIG.25B 



Z X 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 27 of 104 



FIG.26 



C START ) 



DISPLAY OBJECT AND POINTER 



5 



STEP 402 —STEP 404 



405 



THERE IS OBJECT 
THAT IS POINTED AT? 

YEES 



401 



NO 



CHANGE COLOR OF THE POINTED OBJECT AND DISPLAY IT ^406 



OPERATION FOR 
STARTING OPERATION. EDITING, AND PROCESSING. 

IS PERFORMED? 

YES 



411 



HIDE THREE-DIMENSIONAL POINTER 



412 



DISPLAY PROJECTION OF OBJECT TWO-DIMENSIONALLY 
ONTO DISPLAY PLANE CLOSEST TO OPERATOR 



5 



413 



ACCEPT TWO-DIMENSIONAL GUI-LIKE 
OPERATION FOR OBJECT BY OPERATOR 



414 




OPERATION FOR 
.ENDING OPERATION, EDITING, AND PROCESSING, 

IS PERFORMED? 

YES 



MOVE OBJECT IN THE THREE-DIMENSIONAL 
DEPTH DIRECTION AND DISPLAY THE OBJECT 



416 




MOVE OBJECT TO DEPTH POSITION AT WHICH 
THERE IS THE INTERFERING ANOTHER OBJECT 

AND DISPLAY IT, AND EXECUTE ATTRIBUTE 
HELD BY THE INTERFERING ANOTHER OBJECT 



419 



MOVE OBJECT TO PREDETERMINED 
DEPTH POSITION AND DISPLAY IT 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 28 of 104 



FIG.27 



( START ) 



DISPLAY OBJECT AND POINTER 



401 




CHANGE COLOR OF THE POINTED OBJECT AND DISPLAY IT ^406 



operation for 
Starting operation, editing, and processing; 

is performed? 

YES 



411 



412~STE P 415 




CHANGE COLOR OF THE POINTED OBJECT AND DISPLAY IT |^406' 



IS BUTTON 
OF PEN PUSHED WHILE OBJECT IS BEING 

POINTED AT? 



407 



STEP 408— STEP 410 



417 

THERE 

IS INTERFERING ANOTHER OBJECT 

9 



NO 



DISPLAY THE INTERFERING ANOTHER 
OBJECT AFTER CHANGING THE COLOR 



^ 421 
BUTTON^ 

OF PEN IS PUSHEQ. 

? 



420 



NO 



418 



EXECUTE ATTRIBUTE 
HELD BY THE INTERFERING 
ANOTHER OBJECT 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 29 of 104 




FIG.28B 



Obion, Spivak, et al. 
' 703-413-3000 
Docket # 293352US40PCT 
Sheet 30 of 104 



FIG.29A 




FIG.29B 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 31 of 104 



FIG.30A 




z 

^ X 



FIG.30B 



201 



201 D 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 32 of 104 



FIG.30C 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 33 of 104 



FIG.31B 



302 





Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 34 of 104 



FIG.32B 



X 




FIG.32C 



x 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 35 of 104 



FIG.33 




FIG.34 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 36 of 104 



FIG.35A 




FIG.35B 



201 



301 




303 



Obion, Spivak, et al 

703-413-3000 

Docket U 293352US40PCT 

Sheet 37 of 104 



FIG.35C 



201 




FIG.36A 



301 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 38 of 104 



FIG.36B 




FIG.36C 



301 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 39 of 104 



FIG.37A 




302a 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 40 of 104 



FIG.37C 




303 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 41 of 104 



FIG.38B 




FIG.38C 



201 D 

302c 



201 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 42 of 104 



FIG.39A 



X 





302a 







FIG.39B 




Track:01 





201 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 43 of 104 



FIG.39C 



x 




FIG.40A 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 44 of 104 



FIG.40B 



Z X 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 45 of 104 



FIG.41 



INPUT 
APPARATUS 



X 



1 



INPUT INFORMATION 
OBTAINING MEANS 



101 



INPUT INFORMATION 
PROCESS MEANS 



109 



POINTER POSITION/ROTATION 
ANGLE CALCULATION MEANS 



102 



x 



104 



POINTING 
DETERMINATION 
MEANS 



X 



103 



X 



107 



PROCESS 
CONTROL MEANS 



POINTER 
GENERATION 
MEANS 



X 



105 



OBJECT 
GENERATION 
MEANS 



X 



106 



DISPLAY CONTROL MEANS 




STORING 
MEANS 



DISPLAY 
APPARATUS 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 46 of 104 



FIG.42 



^j— 201 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 47 of 104 



FIG.43 



( START ) 
3 



DISPLAY OBJECT 



4 



401 



OBTAIN STATE OF PEN 



402 




403b 

BUTTON^ 
201 D2 IS PUSHED 



OUTPUT INFORMATION 
INDICATING THAT BUTTON 201 Di IS 
PUSHED TO CALCULATION MEANS 



OUTPUT INFORMATION 
INDICATING THAT BUTTON 201 D2 IS 
PUSHED TO CALCULATION MEANS 



I 



CALCULATE POSITION, ORIENTATION, 
LENGTH, ETC. OF POINTER 



I 



405 



DISPLAY POINTER AT THE CALCULATED POSITION WITH THE 
CALCULATED ORIENTATION AND THE CALCULATED LENGTH 



406 



NO 



IS THERE 
OBJECT WITHIN 10 PIXEL FROM 
TIP OF THE POINTER? 

YES 



407 



CHANGE COLOR OF THE OBJECT 
THAT IS POINTED AT AND DISPLAY IT 



SELECT OR HOLD OBJECT 



408 




410 



( END ) 



Obion, Spivak, et al. 
703-413-3000 
Docket # 293352US40PCT 
- Sheet 48 of 104 




3 



Obion, Spivak, et al 
703-413-3000 

Docket # 293352US40PCT 
Sheet 49 of 104 



FIG.45 



420 



421 



422 



423 



424 



425 



c 



START 



I 



PERFORM OPERATION OF SELECTING OR 
HOLDING OBJECT BY THE PROCEDURE OF 
EMBODIMENT 2-1 OR EMBODIMENT 2-2 



OBTAIN STATE OF PEN 



CALCULATE POSITION, ORIENTATION, 
LENGTH ETC. OF POINTER AND OBJECT 



DISPLAY POINTER AND OBJECT AT THE CALCULATED 
POSITION WITH THE CALCULATED ORIENTATION 
AND THE CALCULATED LENGTH 




RELEASE THE STATE OF SELECTING OR HOLDING 
OBJECT 

DISPLAY OBJECT AT A POSITION AND IN A STATE 
JUST BEFORE BUTTON 201 Ds IS PUSHED 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 50 of 104 



FIG.46 



C START ) 

PERFORM OPERATION OF SELECTING OR 
HOLDING OBJECT BY THE PROCEDURE OF 
EMBODIMENT 2-1 OR EMBODIMENT 2-2 

DELETE DISPLAY OF POINTER 

1 

DISPLAY OBJECT AS TWO-DIMENSIONAL 
IMAGE ON THE NEAREST SID E PLANE 

1 

ACCEPT TWO-DIMENSIONAL GUI-LIKE 
OPERATION, EDITING, AND PROCESSING 




YES 

I 

MOVE OBJECT TO A DESIRED POSITION 

BY THE PROCEDURE OF 
EMBODIMENT 2-3 OR EMBODIMENT 2-4 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 51 of 104 



420 



426 



427 



428 



429 



431 



432 



433 



FIG.47 



C START ^ 



PERFORM OPERATION OF SELECTING OR 
HOLDING OBJECT BY THE PROCEDURE OF 
EMBODIMENT 2-1 OR EMBODIMENT 2-2 



5 



DELETE DISPLAY OF POINTER 



I 



DISPLAY OBJECT AS TWO-DIMENSIONAL 
IMAGE ON THE NEAREST SIDE PLANE 



I 



ACCEPT TWO-DIMENSIONAL GUI-LIKE 
OPERATION, EDITING, AND PROCESSING 



MOVE OBJECT IN THE DEPTH DIRECTION 
UNTIL IT INTERFERES WITH THE ANOTHER 
OBJECT, AND DISPLAY THE OBJECT 



I 



EXECUTE ATTRIBUTE OF 
THE INTERFERING OBJECT 




434 



MOVE OBJECT TO 
PREDETERMINED 
DEPTH POSITION AND 
DISPLAY THE OBJECT 



( END ) 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 52 of 104 



FIG.48 




Obion, Spivak, et al. 
703-413-3000 

Docket # 293352US40PCT 
Sheet 53 of 104 



FIG.50 



INPUT 
APPARATUS 



X 



1 



£ 

INPUT INFORMATION 
OBTAINING MEANS 



101 



102 



POINTING POSITION/ 
DEFORMATION AMOUNT 
CALCULATION MEANS 



107 



X 



108 



PROCESS 
CONTROL 
MEANS 



x 



105 



POINTING 
DETERMINATION 
MEANS 



X 



106 



OBJECT 
GENERATION 
MEANS 



X 



103 



STORING 
MEANS 



POINTER 
GENERATION 
MEANS 



DISPLAY 
CONTROL 
MEANS 



104 



DISPLAY 
APPARATUS 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 54 of 104 



FIG.51 




201A 



202A 



202 



202A 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 55 of 104 



FIG.52 



301 



(xp,y P ,zp) 





+ WHEEL(+Z) 



301 



\ 



Z Y 





+ WHEEL (+Z) 



301 




301 



-0 » 



Y Z 



301 




Y Z 



301 



J 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 56 of 104 



FIG.53 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 57 of 104 



FIG.54 



301 



(xp,y P> z P ) 




301 




+ WHEEL(-Z) 




301 




+ WHEEL(-Z) 



i 



■ • • 




i ; 4 



X 



X 



Obion, Spivak, et aL 

703-413-3000 

Docket # 293352US40PCT 

Sheet 58 of 104 



FIG.55 



c 



START 




CALCULATE MOVING DIRECTION, 
MOVING AMOUNT, ETC. OF POINTER 



603 



DISPLAY POINTER AT NEW DISPLAY POSITION 



I 



CALCULATE INCLINATION DIRECTION, 
INCLINATION ANGLE, ETC. OF POINTER 



I 



DISPLAY POINTER BY SLANTING IT 



YES 



604 



605 



606 




NO 



CHANGE COLOR OF THE OBJECT 
THAT IS POINTED AT AND DISPLAY IT 



608 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 59 of 104 



FIG.56A 



(xp,y P ,Zp) 




4A 



FIG.56B 



(xp.yp.zp) 




FIG.56C 



(x P ,y P ,Zp) 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 60 of 104 



FIG.56D 



(x P ,y P ,Zp) 




4D 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 61 of 104 



FIG.57 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 62 of 104 



FIG.58 



301 



(x Pf y Pt zp) 




301 




+ WHEEL (+Z) 





+ WHEEL (+Z) 



301 



i, 






301 




301 





Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 63 of 104 



FIG.59 



301 



301 



(x P ,y P ,zp) 





+ WHEEL(+Z) 



\ 





4 




+ WHEEL (+Z) 



301 



- ■ ■ • • • * .* 4 * « * 



.... 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 64 of 104 



FIG.60 



(x Pl y P ,Zp) 




\ 



+ WHEEL(+Z) 




+WHEEL(+Z) 




(xp,y Pf z P ) 



Obion, Spivak, et al. 
703-413-3000 

Docket # 293352US40PCT 
Sheet 65 of 104 



FIG.61 




301 B 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 66 of 104 



FIG.62A 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 67 of 104 



FIG.62B 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 68 of 104 



FIG.63A 




FIG.63B 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 69 of 104 



FIG.64A 




Y Z 



FIG.64B 



301 




Y Z 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 70 of 104 



FIG.65A 



PATH OF (x P> y P , z P ) 



301 




Y Z 



FIG.65B 




z 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 71 of 104 



FIG.66A 



PATH OF (x P , y P . z B ) 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 72 of 104 



FIG.67 



(x P ,y P ,Zp) 




301 



Ctrl 



+ WHEEL (+Z) 




301 



Ctrl 



+ WHEEL(+Z) 



® ► 





301 





(xc,y c ,zc) 




— i 



(xc,yc,zc) 



301 




I (xc F yc r z c ; | 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 73 of 104 



FIG.68A 



301 




Y Z 



FIG.68B 




Y Z 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 74 of 104 



FIG.69 



301 





FIG.70A 



301 




Y Z 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 75 of 104 



FIG.70B 




Obion, Spivak, et al. 
703-413-3000 

Docket # 293352US40PCT 
Sheet 76 of 104 



FIG.71 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 77 of 104 



FIG.72 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 78 of 104 



FIG.73 



c 



START 



I 



DISPLAY OBJECT AND POINTER 



5 



OBTAIN INPUT INFORMATION ON POINTER 



I 



CALCULATE MOVING DIRECTION, 
MOVING AMOUNT, ETC. OF POINTER 



I 



DISPLAY POINTER AT NEW DISPLAY POSITION 



CALCULATE MOVING DIRECTION AND 
MOVING AMOUNT OF POINTING PART 



DISPLAY POINTER IN WHICH ONLY POINTING 
PART IS MOVE, AND THE POINTING PART AND 
NON-POINTING PART ARE CONNECTED 



607 



601 



602 



603 



604 



609 



610 



THERE IS OBJECT 
THAT IS POINTED AT?^ 



NO 



YES 



CHANGE COLOR OF THE OBJECT 
THAT IS POINTED AT AND DISPLAY IT 



608 



FIG.74A 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 
Sheet 79 of 104 



FIG.74B 




FIG.74C 



z 




FIG.74D 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 80 of 104 



FIG.75 




Y Z 



FIG.76A 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 81 of 104 



FIG.76B 




Y 



FIG.77A 




Y Z 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 82 of 104 



FIG.77B 




5 



Z 



FIG.78A 



PATH OF (x P , y P , z P ) 



301 




Y 



Obion, Spivak, et al 

703-413-3000 

Docket # 293352US40PCT 

Sheet 83 of 104 



FIG.78B 



301 



PATH OF 

(x P , y P , z P ) 




(xc, y c> zc) 







FIG.79A 



301 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 84 of 104 



FIG.79B 



301 




FIG.79C 



301 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 85 of 104 



FIG.80 



INPUT 
APPARATUS 



X 



1 



X 



101 



INPUT INFORMATION 
OBTAINING MEANS 



X 



102 



POINTER POSITION 
CALCULATION 
MEANS 



107 



X 



108 



PROCESS 
CONTROL MEANS 



104 



OBJECT CHANGE 
DETERMINATION 
MEANS 



STORING 
MEANS 



X 



105 



OBJECT GENERATION 
/TRANSPARENTIZING 
MEANS 



X 



103 



POINTER 
GENERATION 
MEANS 



DISPLAY 
CONTROL MEANS 



104 



DISPLAY 
APPARATUS 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 86 of 104 



FIG.81 




4A 

(BRIGHTNESS La) 



301A 




301B 



4B 

(BRIGHTNESS 
Lb) 



Lb 



La 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 87 of 104 



CO CM CM CM CO CM 




_Q -o 

CM CM 



Obion, Spivak, et al 
703-413-3000 

Docket # 293352US40PCT 
Sheet 88 of 104 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 89 of 104 



FIG.84 




Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 90 of 104 



FIG.85 



402a 




402b 



403 



401 








* - 
1 


f I 

<V — — 

i I 


1 

I _ 





MOVE POINTER 
TOWARD z>0 DIRECTION 




401 



301 A 




402a 



402b 



301 B 



MOVE POINTER 
TOWARD z<0 DIRECTION 



403 



'.'.V. V*'< 

hJOmi lilt 




'iwwiffiifi 



401 



MOVE POINTER 
TOWARD z>0 DIRECTION 





401 




I 



X 



403 



MOVE POINTER 
TOWARD z<0 DIRECTION 




403 





402b 



401 



403 



401 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 91 of 104 



FIG.86 



402a 



401 




402b 


403 










> 

r - 

1 

1 

I - 


f 1 

N 

1 

1 

1 



MOVE POINTER 
TOWARD z>0 DIRECTION 



A 




401 



301 A 




402a 



402b 



301 B 



MOVE POINTER 
TOWARD z<0 DIRECTION 



□□□ 



401 



403 



9 



_ « 1 m r * F 1 



j?T ^-402b 



f : 



403 




402b 



401 



MOVE POINTER 
TOWARD z>0 DIRECTION 



MOVE POINTER 

TOWARD z<0 DIRECTION 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 92 of 104 



FIG.87 



402a 




401 



402b 



403 





1 □□□ 


\ 


t 1 


r - 

i 

i 

I _ 


" v 

! 
1 

i 





403 



301 A 




402a 



402b 
401 



301 B 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 93 of 104 



FIG.88 



504 



I 



PERFORM 
PROCESS 
ACCORDING 
TO OBTAINED 
INFORMATION 



C 



START 



I 



) 



DISPLAY POINTER AND OBJECT 



5 



OBTAIN INFORMATION 
FROM INPUT APPARATUS 



503 



OBTAIN 

INFORMATION ON OPERATION 

OF POINTER? 

YES 



CALCULATE MOVING AMOUNT OF POINTER 
FROM OPERATION INFORMATION TO CALCULATE 
DISPLAY POSITION (x P ,yp,z P ) OF POINTER 



I 



GENERATE POINTER TO DISPLAY AT 
THE CALCULATED DISPLAY POSITION 



NO 



507 



THERE IS OBJECT 
THAT IS POINTED AT? 



YES 



DISPLAY THE OBJECT THAT IS 
POINTED AT AFTER CHANGING 
THE OBJECT TO POINTING STATE 




DISPLAY THE FRONT SIDE OBJECT 
WITH TRANSPARENCY Ol 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 94 of 104 



FIG.89 



402b 



403 



402a 




401 



? 4 t \ i 

vMv 



ft 



□□□ 




401 



301A 




402a 



402b 



MOVE POINTER 
TOWARD z>0 DIRECTION 



301 B 



MOVE POINTER 

TOWARD z<0 DIRECTION 



401 




403 



MOVE POINTER 
TOWARD x>0 DIRECTION 



402a 





402b 



401 



MOVE POINTER 
TOWARD x<0 DIRECTION 



403 



~ — -• - — — — - — • - 

: Uil— !l I I 


i 

: fw?r r 

* ■ • « • 4 m 

: iv.'.v. 

■ I* * . » • 


•M'l j 

r ^~402b | 




402b 



401 



401 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 95 of 104 



FIG.90 



Q START ^ 



CHECK DISPLAY REGION OF THE OBJECT 
IN THE FRONT SIDE OF POINTER 




I YES 

i 



DISPLAY THE OBJECT OVERLAPPING 
THE POINTER WITH TRANSPARENCY a 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 96 of 104 



FIG.91 



402a 




402b 



403 



401 



□□□ 



* -s. 



I I 



MOVE POINTER 
TOWARD z>0 DIRECTION 




401 



301 A 




402a 



402b 



301 B 



MOVE POINTER 
TOWARD z<0 DIRECTION 



402b 



403 



403 




402b 



401 



MOVE POINTER 
TOWARD z>0 DIRECTION 



MOVE POINTER 
TOWARD z<0 DIRECTION 



402a 





403 





402a 
402b 



401 



403 



401 



Obion, Spivak, et aL 

703-413-3000 

Docket # 293352US40PCT 

Sheet 97 of 104 



FIG. 9 2 



Of=1 : COMPLETELY OPAQUE 

Of=1 : COMPLETELY TRANSPARENT 




Z DISTANCE BETWEEN OBJECT 
IN THE FRONT SIDE OF POINTER AND THE POINTER 



FIG.93 



c 



START 




CALCULATE DISTANCE BETWEEN OBJECT 
TO BE TRANSPARENTIZED AND POINTER 



I 



CALCULATE TRANSPARENCY a OF 
OBJECT BASED ON CALCULATED DISTANCE 



DISPLAY OBJECT WITH CALCULATED 

TRANSPARENCY a 



510d 



51 Oe 



51 Of 



51 Og 



51 Oh 



THERE IS ANY 
OTHER OBJECT TO BE 
TRANSPARENTIZED? 



YES 



NO 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 98 of 1 04 



FIG.94 



402a 




402b 



403 



i a * • 

. • f • ¥ ■ 

► 

« 4 • » « « 4 

t »»»»»» 



401 





1 □□□ 


1 
1 

I. 


f I 

' N . , 

1 

f 

I 





MOVE POINTER 
TOWARD z>0 DIRECTION 



403 



MOVE POINTER 
TOWARD z>0 DIRECTION 






401 



301A 




402a 



402b 



301 B 



MOVE POINTER 
TOWARD z<0 DIRECTION 




401 




I 



X 



403 



MOVE POINTER 
TOWARD z<0 DIRECTION 



4 • 4 4 • 4 * 



402b 



□□□ 


_ » * ■ * * j 
1 ^'. € * * • * A 

m /r »»■»•«■ 
A 1 ■ • • 4 « 4J 






J 1 



403 




402b 



401 



403 



401 



(x P , y P ) 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 99 of 104 



FIG.95A 



Qf=1 : COMPLETELY OPAQUE 

01=0 : COMPLETELY TRANSPARENT 




i \ 1 / 





xo=xp-a xo=xp xo=xp+a MAXIMUM 

X COORDINATE (xo) OF OBJECT 
IN THE FRONT SIDE OF POINTER 



FIG.95B 



Qf=1 : COMPLETELY OPAQUE 

a=0 : COMPLETELY TRANSPARENT 







Xo— Xp 



MAXIMUM 



X COORDINATE (x 0 ) OF OBJECT 
IN THE FRONT SIDE OF POINTER 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 100 of 104 



FIG.96 



402a 




402b 



403 



401 





□□□ 



v 



t » 



401 



301 A 




402a 



402b 



MOVE POINTER 
TOWARD z>0 DIRECTION 



301 B 



402b 



403 




401 




402b 



403 



POINTER STOPS FOR 
A PREDETERMINED TIME 




402b 



□□D 


✓ - v 

i i 

. — i 


•> 




403 




402b 



401 



403 



401 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 101 of 104 



FIG.97 



504 

_2_ 



( START ) 



DISPLAY POINTER AND OBJECT 



PERFORM 
PROCESS 
ACCORDING 
TO OBTAINED 
INFORMATION 



501 



OBTAIN INFORMATION FROM INPUT APPARATUS 



502 



503 



OBTAIN 

INFORMATION ON OPERATION 

OF POINTER? 



YES 



STEP 505' 


-STEP 510 







511 



YES 



OBTAINING OF 
INFORMATION ON 
OPERATION OF POINTER IS 
CONTINUING? 



NO 



PREDETERMINED 
TIME HAS EXPIRED AFTER 
OBTAINING OF INFORMATION ON 
OPERATION OF POINTER 
STOPS? 

YES 



512 



NO 



RESTORE TRANSPARENTIZED OBJECT 
TO OPAQUE AND DISPLAY THE OBJECT 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 102 of 104 



FIG.98 



403 




SELECT OBJECT 



MOVE SELECTED OBJECT 
TOWARD FRONT SIDE 



n 



403 




301A 




402a 



402b 



401 



301 B 





401 



±^>i 402a 

402b 




402a 



403 



403 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 103 of 104 



FIG.99 



403 



402a 




MOVE POINTER IN SELECTED STATE 



RELEASE SELECTION 



402b 
401 




402a 



403 




402b 



401 






402b 



401 



\ 



402a 




403 



Obion, Spivak, et al. 

703-413-3000 

Docket # 293352US40PCT 

Sheet 104 of 104 



FIG.100 



c 



START 




PERFORM 
PROCESS 
ACCORDING 
TO OBTAINED 
INFORMATION 



t 



OBTAIN INFORMATION FROM INPUT APPARATUS 



502 



503 



OBTAIN 

INFORMATION ON OPERATION 
OF POINTER? 



YES 



CALCULATE MOVING AMOUNT OF POINTER 
FROM OPERATION INFORMATION TO CALCULATE 
DISPLAY POSITION (xp.yp.zp) OF POINTER 



505 



I 



GENERATE POINTER TO DISPLAY AT 
THE CALCULATED DISPLAY POSITION 



506 



TO 



514 

THERE IS 
CANDIDATE OBJECT 
TRANSPARENTIZED IN THE 
FRONT SIDE OF 
POINTER ? 



NO 



YES 



517 



2 



515 

THERE IS SELECTED 
OBJECT OF CANDIDATE OBJECTS TO 
BE TRANSPARENTIZED? 



NO 



TRANSPARENTIZE ALL CANDIDATE 
OBJECTS AND DISPLAY THEM 



TRANSPARENTIZE ONLY 
NON-SELECTED OBJECTS 
AND DISPLAY THEM 



516 



